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* NOTICES * 

Japan Patent Office is not responsible for any damages caused by the use of this 
translation. 

1. This document has been translated by computer. So the translation may not reflect the 
original precisely. 

2 **** shows the word which can not be translated. 
3. In the drawings, any words are not translated, 

CLAIMS 



[Claim(s)] 

[Claim 1] A selection means to choose the 1st image information from the image 
information for ** (n+1) followed from the m-th frame (m+n) to the frame eye (for m and 
n to be the natural number of arbitration), The storage means for arranging and 
memorizing the image information of the above-mentioned (n+1) **, An operation means 
to calculate the motion vector to the 1st image information, respectively about image 
information of n sheets other than the image information of the above 1 st, The image 
processing system characterized by establishing an arrangement means to arrange the 
image information of the n+1 above-mentioned sheet, respectively, and a synthetic 
means to compound the image of ** (n+1) after the above-mentioned arrangement, and 
to form the image of one sheet, based on the above-^mentioned result of an operation. 
[Claim 2] The above-mentioned selection means is an image processing system 
according to claim 1 characterized by having an evaluation means to evaluate the 
characteristic quantity of the image of the above-mentioned (n+1 ) **, respectively, and 
choosing one sheet from ** (n+1 ) based on the evaluation result. 

[Claim 3] The above-mentioned characteristic quantity is an image processing system 
according to claim 2 characterized by being the edge information on an image. 
[Claim 4] The above-mentioned evaluation means is an image processing system 
according to claim 2 characterized by what it has a filter means to extract the edge of an 
image, and is evaluated based on the edge extract information after the filtering. 
[Claim 5] The above-mentioned selection means is an image processing system 
according to claim 1 characterized by choosing based on the entry sequence of the 
image of the above-mentioned (n+1) **. 

[Claim 6] The above-mentioned selection means is an image processing system 
according to claim 5 characterized by choosing the frame equivalent to the middle time 
of day of the above-mentioned entry sequence. 

[Claim 7] The selection procedure which chooses the 1 st image information from the 
image information for ** (n+1) followed from the m— th frame (m+n) to the frame eye (m 
* and n are the natural number of arbitration), The operation procedure of calculating the 
motion vector to the 1st image information, respectively about image information of n 
sheets other than the image information of the above 1st, The image-processing 
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approach characterized by fornning the arrangement procedure which arranges the image 
information of the n+1 above-mentioned sheet, respectively, and the synthesis 
procedure which compounds the image of ** (n+1) after the above-mentioned 
arrangement, and forms the image of one sheet based on the above-mentioned result of 
an operation. 

[Claim 8] The selection processing which chooses the 1st image information from the 
image information for ** (n+1) followed from the m-th frame (m+n) to the frame eye (m 
and n are the natural number of arbitration), Data processing which calculates the motion 
vector to the 1st image information, respectively about image information of n sheets 
other than the image information of the above 1st, The storage which memorized the 
program for performing arrangement processing which arranges the image information of 
the n+1 above-mentioned sheet, respectively, and synthetic processing which 
compounds the image of ** (n+1) after the above-mentioned arrangement, and forms the 
image of one sheet based on the above-mentioned result of an operation and in which 
computer reading is possible. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] Especially this invention relates to image output units, such as a 
printer which carries out expansion variable power of the inputted image information, and 
outputs it, and the storage which is the communication link between models from which 
resolution differs, uses when carrying out resolution conversion from low resolving 
information at high resolving information, and is used for a suitable image processing 
system, an approach, and them and in which computer reading is possible. 
[0002] 

[Description of the Prior Art] Various approaches are proposed as an approach of 
conventionally carr3dng out resolution conversion of the low resolving information on the 
inputted image at high resolving information. The transform-processing approach changes 
with classes (for example, the multiple-value image which gradation information has for 
every pixel, the binary image made binary by false halftone, the binary image made binary 
by the fixed threshold, an alphabetic character image, etc.) of target image [ approaches / 

« 

these / conventional ]. 

[0003] Generally the primary ** interpolation method which determines the pixel value E 
by the following operations is used with the distance surrounding a interpolation point as 
shown in the maximum contiguity interpolation approach of arranging the same pixel 
value nearest to a interpolation point as shown in drawing 12 as the conventional 
interpolation approach, and drawing 13 of four points (the pixel value of four points is set 
to A, B, C, and D). 

E= (1-i) -(1-j)- A+i .(1-j)- B+ (1-i) and j-C +i-j-D .... (1) 
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(However, when distance between pixels is set to 1, suppose that the distance of j is in i and 
a perpendicular direction horizontally from A (i<=l, j<=l)) . 

[0004] Moreover, it is reproducible as a means to change the sampled discrete signal into a 
continuous ringing, by passing the ideal low pass filter which can be expressed with a 
SING function as expressed with the sampling theorem for many years. Since the 
processing time starts, calculating a SING function approximates the interpolation 
function expressed with a SING function, and it has the approach of computing a 
interpolation value only by the easy sum-of-products operation. 

[0005] "image-analysis handbook: According to Mikio Takagi and Akihisa Shimoda 
editorial-supervision University of Tokyo Press", in a 3rd convolution interpolation method 
(Gubic Gonvolution interpolation), approximation of a interpolation function is reahzable. 
The image data for which it asks is interpolated using the 3rd convolution function shown 
by the following formula using the image data of 16 stations around a point to interpolate. 
[0006] 

[Equation 1] 



^•'[/(yl) /(y2) /{r3) /(y4)] 



Pll F12 P13 P14 

F21 P22 P23 P2A 

F31 P32 P33 P34 

F41 P42 P43 JP44 



f{x3) 



(2) 



[0007] 

[Equation 2] 



[0008] 

[Equation 3] 



[0009] In addition, Pn-P44 of a formula (2) shows a circumference pixel value, and shows 
arrangement to drawing 14 . 

[0010] However, in three kinds of conventional examples mentioned above, the jaggy of the 
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letter of a block which was all dependent on the dotage and input low resolving by 
interpolation at the time of interpolation occurred, and high-definition high resolving 
information was not able to be created. Then, these people proposed the approach which 
can perform resolution conversion, without there being also no interpolation dotage by 
interpolation processing, and a jaggy occurring by JP,7-93531A> JP,7-107268,A, 
JP,7-105359,A, etc. in creation of low resolving information to high resolving information. 
[0011] The fundamental view of these proposals is in the condition which removed the 
resolution dependence component from the inputted original information, and was 
removed, and is the approach , of guessing and creating the information corresponding to 
new resolution in the inside in the condition of having made it increasing to an equivalent 
for output resolution, and having made the number of pixels increasing. As a means which 
removes the dependency of input resolution, the increment in smoothing by LPF and the 
number of pixels is realizable with linear interpolation. The guess of high resolving 
information computes the pixel value to output by making information after interpolation 
simple binary and performing processing which is different to the pixel classified into "1", 
and the pixel classified into "0", respectively. 

[0012] Moreover, as proposed by JP,9-252400,A, there is also the approach of creating the 
good edge at which the continuity of a pixel value was maintained. In this official report, 
the pixel (however, the pixel value in the station n in m points is set to P (n)) of m points 
(m>=l) is detected from the circumference pixel of a low resolution attention pixel, and 
output- value h (k) is calculated by the following formulas based on said interpolation value 
C in each interpolating point k of having interpolated the attention pixel to two or more 
pixels (k). 
[0013] 

[Equation 4] ^ 



[0014] 

[Problem(s) to be Solved by the Invention] However, there was a fault which is described 
below in the above-mentioned conventional example. That is, however it may create high 
resolving information, it is the point that there is a limit in high definition-ization. Since 
the information beyond the nyquist limitation of input resolution naturally does not exist 
in an input image so that more clearly than a sampling theorem, all information creation 
more than a Nyquist rate is based on a guess. 

[0015] Therefore, although it is easy to change into jaggy loess a flat artificial image Hke 
CG image which is not not much compUcated, an illustration image, and an animation 
image, high-definition-izing by the information guess beyond the nyquist limitation of a 
natural image is difficult. That is, even if it uses what kind of approach, as compared with 
the image into which the image which inputted low resolving information and was changed 
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into high resolving inputted high resolving information from the first, image quality 
becomes low clearly. 

[0016] On the other hand, the means which can input the photoed d3niamic image into a 
computer per one continuous frame has been increasing by the spread of digital 
camcorders etc. in recent years. However, although the output resolution of a printer is 
increasing every year, and the input resolution of an image pick-up system is increasing, 
compared with printer resolution, the present condition is still low. 

[0017] Then, the high resolving still picture of one sheet is not created from a low resolving 
still picture of one sheet which was stated with the technique of the conventional example, 
but the technique which creates the high resolving still picture of one sheet is proposed by 
this invention from two or more continuous low resolving still pictures incorporated from 
the animation. Synthetic : Yoshizawa of a background image who took paiming of 
"dynamic image into consideration as the creation approach of a wide range panorama 
image from two or more still pictiires conventionally, Hanamura, Tominaga, collection of 
********** size drafts 7-51(1990)", and the method of generating the panorama image by 
"division image pick-up: There is a proposal by collection of Nakamura, Kaneko, wood, and 
********** size drafts 7-165(1991)" etc. 

[0018] However, more a few [ the panorama image to which the image pick-up range was 
expanded is not created, but the image pick-up range is the same, and / the proposal of the 
technique which is made to compound the information on two or more still pictures, and 
raises the resolution of an image by interpolation ] than the still picture of one sheet. 
[0019] As a technique of still picture creation of high resolving, there is a proposal by 
JP,5-260264,A from such an animation of low resolving. This proposal compares the 
continuous images, from the difference in a two-sort image, detects the parameter of a£&ne 
transformation and a parallel displacement, and compounds a two-sort image. The 
example which uses composition for interpolation is stated to the 2nd example of the 
above-mentioned proposal. 

[0020] However, there are the following troubles by the above-mentioned proposal. That is, 
by comparing the sequential images expanded by the interpolation approach shown in 
drawing 14 from drawing 12 mentioned above, the approach indicated by the 2nd example 
of the above-mentioned official report computes the parameter mentioned above, and 
determines and compounds the interpolation location. However, since the interpolation 
operation itself is not what creates new high resolving information, the exact decision of 
the coordinate to compound is also difficult. 

[0021] I hear that interpolating interpolates between pixels and there is. When comparing 
sequential images by the above-mentioned approach, there is no information between the 
pixels of input resolution. If it puts in another way simply and two sorts of images will be 
assumed to be Image A and Image B, the decision in which location between the pixels of 
Image A to interpolate the pixel of Image B is difficult in the comparison between mere 
expansion images. 

[0022] The smallest unit of the vector quantity of a motion vector is a pixel unit, and this 
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originates in the point that there is no resolution finer than the distance between pixels. If 
resolution-of-a-vector ability does not have the precision below between pixels, it becomes 
namely, almost unchanging [ in / as the case of the creation to the high resolving still 
picture of one sheet of one sheet which the effectiveness of interpolating using two or more 
still pictures faded, and was stated in said conventional example from a low resolving still 
pictiire / image quality ] less. 

[0023] The basic technique of "international-standards image coding: Fumitaka Ono and 
Watanabe ** Corona Publishing" has explanation of some approaches about the various 
motion detecting methods. However, since it differs &om the purpose of this invention 
which is going to create the image of one sheet from two or more images, even if it uses 
these techniques, composition of good two or more images is difficult [ any approach of this 
explanation is the detection approach aiming at a motion compensation, and / a fine 
detection precision is unnecessary, and ]. 

[0024] Then, these people proposed the calculation approach of a vector with resolving 
power finer than the distance between pixels using orthogonal transformation. By this 
approach, two or more still pictures are not what became independent respectively, and 
the injury with relation of the spatial coordinate between images of them was attained. 
However, even if it moves and the resolution of ** KUTORU is computed more finely than 
the distance between pixels, many troubles still remain by the ability of the relative 
position between images to only grasp correctly. 

[0025] One of them is the problem that the error at the time of compounding causes image 
quality degradation. That is, if the number of sheets of the image to compound increases, 
the error of a motion vector will be accumulated indeed. 

[0026] Drawing 15 is drawing having shown the sequence which computes the 
conventional motion vector. In drawing, the example which compounds the image for four 
frames with which it continued from m time of day (m+3) to the frame is explained. 1501, 
1502, 1503, and 1504 show the image information of the m-th frame, the image 
information of a fi:ame (m+1) eye, the image information of a frame (m+2) eye, and the 
image information of a frame (m+3) eye, respectively. 

[0027] As sequence which computes a motion vector, it becomes the m-th firame, a frame 
(m+1) eye then a frame (m+1) eye and a frame (m+2) eye, and three vector calculation 
continuously called a firame (m+2) eye and a frame (m+3) eye first. That is, it is the 
approach of computing serially the amount of motions after one-frame advance Uke the 
motion compensation of the conventional dynamic-image coding. In this case, as for an 
aperture, only the amount of [ the target frame (an object firame is called) and ] one firame 
is [ the firame (a criteria frame is caUed) used as the criteria which compute a motion 
vector ] always in time. 

[0028] However, by this approach, there are some problems in the purpose of a motion 
compensation by the composition of two or more images made into the purpose of this 
invention also by the optimal approach. One of them is are recording of the 
above-mentioned error. That is, if the movement magnitude for one frame is not computed 
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correctly, by the motion vector calculation after it, movement magnitude will newly be 
calculated from the image which the error produced in time. The more there is much image 
number of sheets, the error accumulated may increase and, the more a greatly different 
result from an original arrangement location may be brought. 

[0029] Moreover, it is management when a frame with difficult calculation of a motion 
vector exists in the middle of the sequential image of two or more sheets as another 
problem. Naturally, in the sequential image of two or more sheets, the frame of an 
indistinct image also exists by blurring by the side of input devices, such as a video camera 
in the middle of an image pick-up, migration of an object, etc. In that case, if a conventional 
approach hke drawing 15 is used, and there will also be a possibility of making a motion 
vector into what was greatly mistaken as a result and it will make a mistake in vector 
calculation once, as mentioned above, an error will not be canceled to the frame after it. 
[0030] It does not become a problem no matter it may compound two or more sheets in 
what sequence in the appHcation formed into high resolving using the technique of such 
pixel ****** of CCD [ in / it moves and / in the problem of the sequence of ** KUTORU 
calculation / current and the already marketed video camera ]. it is equivalent to the 
relative position for every image -- " it is because it shifts and amoimt" is controlled by 
the device side. However, in the case of the gestalt of operation of this invention mentioned 
later, the relative position for every image is not controlled at all. Therefore, the sequence 
which moves and computes ** E^UTORU brings a big factor to the improvement in image 
quality. 

[0031] Moreover, this invention person proposed the approach of processing image data 
and carrying out arrangement composition so that composition of two or more images 
might be previously fitted to the frame of criteria instead of the approach of only merely 
arranging the pixel value of a multiple frame. By the approach of processing image data, it 
will become the synthetic image with which image quahty completely differs by which 
frame is used as a criteria frame. By the method compounded after data processing, image 
quality differs by which image is used as a criteria frame not only when the number of 
sheets of a sjmthetic frame is three or more sheets, but by the case of two sheets. That is, 
which images were moved by the former as compared with the case where the image of two 
or more sheets is compounded, the amount was calculated, and the control approach of the 
good frame whether to tie to composition was not proposed. 

[0032] Therefore, this invention compoimds the image of two or more sheets, and proposes 
the frame control system in the case of obtaining the image of the high resolution of one 
sheet. 
[0033] 

[Means for Solving the Problem] In the image processing system according to this 
invention in order to attain the above-mentioned purpose A selection means to choose the 
1st still picture from the dynamic-image information for ** (n+1) (for m and n to be the 
natural number of arbitration) that it continued from the m-th frame (m+n) to the frame 
eye, The storage means for arranging and memorizing the dynamic image of the 
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above-mentioned (n+1) **, and the 1st arrangement means which arranges the 1st still 
pictvire by which selection was made [ above-mentioned ] in the above-mentioned storage 
means. An operation means to calculate the motion vector to the 1st still picture, 
respectively about still pictures of n sheets other than the still picture of the above 1st, 
Based on the above-mentioned result of an operation, the 2nd arrangement means which 
arranges the still picture of the n above-mentioned sheets, respectively, and a synthetic 
means to compound the image of ** (n+1) after the above-mentioned arrangement, and to 
obtain the image of one sheet are formed in a different location from the arrangement 
location of the 1st still picture of the above within the above-mentioned storage means. 
[0034] Moreover, it sets to the image-processing approach by this invention. The selection 
procedure which chooses the 1st still picture from the dynamic-image information for ** 
(n+1) that it continued from the m-th frame (m+n) to the frame eye. An arrangement 
means to arrange the 1st still picture by which selection was made [ above-mentioned ] in a 
storage means, and the operation procedure of calculating the motion vector to the 1st still 
picture, respectively about still pictures of n sheets other than the still picture of the above 
1st, Based on the above-mentioned result of an operation, the arrangement procedure 
which arranges the still picture of the n above-mentioned sheets, respectively, and the 
synthesis procedure which compounds the image of ** (n+1) after the above-mentioned 
arrangement, and obtains the image of one sheet are formed in a different location from 
the arrangement location of the 1st still picture of the above within the above-mentioned 
storage means. 

[0035] Furthermore, the selection processing which chooses the 1st still picture in the 
storage by this invention from the dynamic-image information for ** (n+1) that it 
continued from the m-th frame (m+n) to the frame eye. The arrangement processing which 
arranges the 1st still picture by which selection was made [ above-mentioned ] in a storage 
means, and data processing which calculates the motion vector to the 1st still picture, 
respectively about still pictures of n sheets other than the still picture of the above 1st, The 
program for performing arrangement processing which arranges the still picture of the n 
above-mentioned sheets, respectively in a different location from the arrangement location 
of the 1st still picture of the above within the above-mentioned storage means, and 
synthetic processing which compounds the image of ** (n+1) after the above-mentioned 
arrangement, and obtains the image of one sheet based on the above-mentioned result of 
an operation is memorized. 
[0036] 

[Embodiment of the Invention] Hereafter, the gestalt of operation of this invention is 
explained with a drawing. Drawing 1 is a block which shows the image processing system 
by the gestalt of operation of this invention. In addition, although it is efficient to provide 
inside image output units, such as a printer connected mainly through the interior of the 
analog video camera which picturizes a dynamic image, or a digital camcorder or a video 
camera, direct, or a computer, and a video printer, as for the image processing system in 
the gestalt of this operation, it is also possible to build as printer driver software for 
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outputting to the application software in the image processing system which serves as a 
middle adapter by connection between a video camera and a printer, or a host computer, or 
a printer. 

[0037] The configuration and actuation of the image processing system of drawing 1 are 
explained. In drawing 1 , 100 shows the input terminal into which the dynamic image 
picturized with the video camera is inputted. The image picturized with the digital 
camcorder is transmitted to a computer, and the gestalt of this operation describes the 
example changed to the resolution of a printer with the appUcation software in a computer. 
[0038] Reproducing the dynamic image photoed by the digital video from a record medium, 
a user sends the incorporation instruction of an image on the scene which he wants. 
Synchronizing with this incorporation instruction, the image information of the multiple 
frame by which the storing section 101 in a computer was followed immediately after the 
incorporation instruction is stored. Now, an incorporation instruction shall be made into 
the m-th time of day, and the image information of** (n+1) from the m-th frame (m+n) to a 
frame eye shall be stored. 

[0039] It judges of which time of day 102 sets up an image as a criteria frame from the 
image information for the frame (n+1) which showed the selection section and was stored. 
Now, let temporarily the criteria frame set up by the judgment result be Frame G. 103 is a 
means for a frame control section being shown and electing two sorts of images which are 
processing objects. Two sorts of images are the frames G whose one is a criteria frame, and 
another is one frame in n frames stored in addition to Frame G (it considers as Frame H). 
[0040] 104 is a means to measure as a vector the movement magnitude which showed 
motion vector operation part and moved partially based on the difference in the two-sort 
image of Frame G and Frame H. 

[0041] 105 shows the arrangement processing section A and is a means to arrange the 
image of the picturized frame G in memory. This memory has the address space more than 
x(inputted number of frame pixels) (perpendicular direction dilation ratio) x (horizontal 
dilation ratio). So, in the arrangement processing section A, arrangement of the pixel 
corresponding to a predetermined dilation ratio is performed. For example, when a 
horizontal direction and a perpendicular direction are twice the dilation ratios of this, the 
perpendicular direction and the horizontal direction will arrange the pixel of Frame G at 
intervals of a pixel. 

[0042] 106 is a means to process it into the pixel value of Frame H so that the data 
processing section may be shown and the image information of Frame G may be suited well. 
[0043] 107 is a means to arrange in the same memory as the arrangement processing 
section A 105 according to the relative vector quantity of Frame G and Frame H which 
showed the arrangement processing section B and were computed by the frame control 
section 103. In a certain case, the stored frame number repeats three or more same 
processings again mentioned above to return and a new frame in the frame control section 
103. However, in processing of the 2nd henceforth. Frame G is immobihzation, and since it 
is already arranged, it does not newly need to arrange the image information of Frame G. 
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Frame H will be updated by the new frame and each down stream processing of the motion 
vector operation mentioned above, data processing, and arrangement will be performed. 
[0044] 108 is the S3nithetic section, compounds the image of two or more sheets arranged in 
the same memory, and makes it the image information of one sheet. 109 is interpolation 
operation part to which the compound image computes the information on the 
interpolation point which has not been buried by the interpolation operation when the 
information on the interpolation point to desired resolution is not buried yet. 110 shows an 
output terminal and the high-resolution-ized image information is transmitted to a printer 
etc. 

[0045] Drawing 2 shows the configuration of the selection section 102 which is the 
description of the gestalt of this operation. Suppose now that the continuous image of a 
total of four sheets from the image of the m-th frame (m+3) to the image of a frame eye was 
stored in the storing section 101 as an example. 201, 202, 203, and 204 show the edge 
extract section, and are a means to extract the edge information included in an image 
based on the stored image information of four sheets, respectively. 

[0046] The example . of the edge extract filter of Laplacian general to drawing 3 is shown. If 
the value after fs (x y) and edge extract processing is now set to ks (x y) for the pixel value 
in the coordinate (x y) on the image of a frame (m+s) eye (however, 0<=s<=3), with the edge 
extract filter of drawing 3 , it will become the following sum-of-products operations. 
[0047] 

ks =(x y) fs+(x-l, y-1) fs+(x y-1) fs 0 [ x+] 1 and y-l+fs-(x-l, y)8fs+ (x y) fs+(x+l, y) fs+(x-l, 
y+1) fs+(x y+1) fs .... (x+1, y+1) (6) 

[0048] In drawing 2 , 205, 206, 207, and 208 are means to integrate the reinforcement of 
the edge which showed the edge on-the-strength evaluation section, and was extracted by 
the edge extract sections 201*204 over the whole image, respectively. If V and the number 
of horizontal pixels are set to H, the edge reinforcement Ps of a frame (m+s) eye will 
compute the number of vertical pixels of the whole image as follows. 
[0049] 

[Equation 5] 



[0050] 209 shows the maximum edge on-the-strength decision section, and determines the 
frame s from which Ps calculated in the edge on-the-strength evaluation sections 205-208, 
respectively serves as max as a criteria frame. That is, the original performance index of 
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edge reinforcement is set up, and the frame estimated that the reinforcement of an edge is 
the largest over the whole image in the stored image of two or more sheets is chosen. 
[0051] It becomes advantageous to set up an image with the large reinforcement of an edge 
as a criteria frame, also when computing the motion vector mentioned later, and also when 
carrying out data processing of other object frames. The evaluation by edge reinforcement 
can also be most assumed that a focus is the image photoed clearly in two or more stored 
sheets. Therefore, the image of object frames other than a criteria frame becomes the role 
which attaches added value to the image quality of a criteria frame, and, also at the lowest, 
the improvement in image quality more than criteria frame independent is guaranteed. 
[0052] Moreover, although the example which processes all frames to juxtaposition was 
described in order that the configuration of drawing 2 might give explanation easy, 
naturally, the edge extract section and the edge on-the-strength evaluation section may be 
single, and the configuration processed perpendicularly may be used. Moreover, by the 
formula (7), it is also possible to calculate calculation of ks' (x y) naturally using the square 
of ks (x y), although the absolute value of ks (x y) was used. 

[0053] Next, the motion vector operation part 104 is explained. Although approaches 
various for many years are proposed as an approach of computing a motion vector, since 
there is no resolution-of-a-vector ability below the distance between pixels, by the 
conventional approach, it is not suitable for the apphcation which performs composition 
and interpolation and changes the animation of low resolving into the still picture of high 
resolution. 

[0054] The detail block diagram of the motion vector operation part 104 by the gestalt of 
this operation to drawing 4 is shown. Two sorts of images transmitted to the storing 
section 101 lost-motion vector operation part 104 of drawing 1 are the frame G which is a 
criteria frame, an object frame, and the becoming frame H. 

[0055] In drawing 4 , the blocking section 401 blocks the image information of Frame H per 
NxN pixel. As for the value of N, various idea **** assume N= 8 as an example. Now, this 
created 8x8-pixel attention block is temporarily called Block A. Next, the orthogonal 
transformation of Block A is calculated in the orthogonal transformation section 402. 
Although the class of orthogonal transformation is not limited, the Hadamard transform 
which can be calculated easily at high speed, DCT (discrete cosine transform) adopted by 
JPEG (Joint Photografic Expert Group) are common. Now, when DCT is made into an 
example, the transform coefficient of two-dimensional DCT of a NxN pixel is [0056], 
[Equation 6] 
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[0057] It comes out and asks. 

[0058] On the other hand, the criteria frame G is blocked by the MxM' pixel unit by the 
MxM' blocking section 403. At this time, the block of a MxM' pixel unit includes the block A 
in Frame H, and the block of the NxN pixel of the same coordinate, and size relation is set 
to M>=N and M>=N (however, the case of M=M*=N is removed). Now, it is assumed that it 
is M=M'=20. That is, the block of 20x20 including the same coordinate as Block A will be 
prepared in Frame G. 

[0059] Next, Block A and the block of the NxN pixel of the same size are created within a 
20x20-pixel block by the NxN blocking section 404. Creation of a block may be started from 
the same coordinate as Block A, and its ** is good for the first time sequentially from the 
edge of a MxM* block. Now, the block of the NxN pixel created within Frame G is 
temporarily called Block B. 

[0060] The orthogonal transformation section 405 carries out orthogonal transformation of 
the created block B like Block A. Naturally, the orthogonal transformation of the 
orthogonal transformation sections 402 and 405 must be the same conversion means. The 
orthogonal transformation multipUer evaluation section 406 evaluates the similarity of the 
transform coefficient based on the orthogonal transformation multiplier of Block A and 
Block B. The sum of the value which multiplied the Lord of DC (direct current) component 
of a block and AC (alternating current) component by the weighting multipUer according to 
a component based on the component of a low frequency region at the difference of each 
multipUer estimates evaluation of similarity. 

[0061] Now, in order to give explanation easy, the coordinate of a block is made to manage 
with the coordinate of the pixel of the upper left which forms a block (the zero of a block of 
the coordinate of this pixel is called hereafter). That is, as shown in drawing 5 , when the 
zero (it is equivalent to the pixel of the slash section) of Block B is set to (a, b), the 
performance index of the similarity of Block A and Block B is [0062]. 
[Equation 7] 



[0063] It comes out and computes. 

[0064] Since correlation of the transform coefficient during a contiguity block becomes low 
so that it becomes a high frequency region, a high frequency region sets up the value of the 
weighting multiplier W (u, v) small. Spatially, since correlation is very high, a coordinate 
transposes the spatial physical relationship of blocks to the similarity of a transform 
coefficient, and is evaluating the transform coefficient of the low frequency region of near 
blocks by the formula (9). Moreover, although the absolute value is used by the formula (9), 
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the evaluation with the same said of the square of difference is possible. 
[0065] Next, the block control section 407 moves 1 pixel (a, b) of zeros of Block B, newly 
creates a block, and repeats the same processing. That is, if N= 8 and M=M'=20 are made 
into an example, since a 8x8-pixel block can be created during [ 13x13 ] a 20x20-pixel block, 
similarity will be repeatedly calculated to a part for the block count. 

[0066] After all ending the scan of Block B in Frame G, R (a, b) which is the 
above-mentioned performance index judges the coordinate (a', b') which becomes min. That 
is, since it can consider that similarity R (a, b) is an error component during Block AB, also 
spatially the block B (the block at this time is called block B*) when taking the minimum 
value of R (a, b) considers that it is Block A and the nearest block, and judges it to be the 
point which Block A moved. 

[0067] However, like the conventional example, the resolving power of a motion vector is a 
1 -pixel unit, and cannot judge the vector below the distance between pixels only now. So, 
with the gestalt of this operation, a motion vector is guessed with resolving power shorter 
than the distance between pixels. 

[0068] The guess approach of a vector is explained below. In an above-mentioned approach, 
the zero of block B* of the &ame G which takes the minimum value of R (a, b) which set to 
(aO, bO) the zero of the block A which is an attention block on the &ame H which is an 
object frame, and mentioned it above is made into (a', b'). In the transform coefficient 
evaluation section 406, although retrieval of block B' was retrieval of a rough pixel unit, 
fine distance extracted around block B' is guessed shortly. 

[0069] namely, - a transform coefficient evaluation - the section -- 406 - **** -- first -- 
spatial " most - being near ** considering - having - a block B -- * -- retrieval - a 
degree - having asked -- a block -- B -- ' from being minute - a gap - an amount a 
guess - ** - saying two - a step - **** - differing - evaluation -- carrying out - 

m 

[0070] Drawing 6 is a flow chart which shows the operations sequence of the 2nd step of 
above-mentioned guesses. Step S601 (henceforth, step abbreviation) compares the 
performance-index result of the formula (7) of the block created on the left of [ 1 pixel ] 
block B', and the block created in the 1-pixel right, respectively. That is, since the zeros of 
block B* are (a\ b'), they evaluate the size of R (a'+l, b') and R (a* - 1 b'). About this R (a'+l; 
b*) and R (a* - 1 b'), since it is computing in the case of the Ist-step similarity evaluation, it 
is desirable to memorize and hold the result of an operation. 

[0071] In S601, if R (a'+l, b') is estimated to be small and it will be estimated as no by S602 
again, it moves to S603. Next, in S602, the block which sets up the block which consists of 
zeros R (a'+l, b') with Block C, and consists of S603 from Zero R (a* - 1 b*) is set up with 
Block C. By S602, Variable c is set as it and coincidence with c= 1, and it is set as them 
withc=-lby S603. 

[0072] Next, in S604, the performance-index result of the formula (7) of the block created 
on 1 pixel of block B' and the block created under 1 pixel is compared shortly, respectively. 
That is, since the zeros of block B' are (a', b'), they evaluate the size of R (a', b'+l) and R (a', 
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b' - 1). Since it is computing also about the performance index of this similarity in the case 
of the Ist'Step similarity evaluation, it is desirable to memorize and hold the result of an 
operation. 

[0073] In S604, if R (a*, b*+l) is estimated to be small and it will be estimated as no by S605 
again, it moves to S606. Next, in S605, the block which sets up the block which consists of 
zeros R (a*, b'+l) with Block D, and consists of S606 from Zero R (a*, b' - 1) is set up with 
Block D. By S605, Variable d is set as it and coincidence with d= 1, and it is set as them 
with d=-l by S606. 

[0074] next -- S -- 607 **** — a block — A— orthogonal transformation — a multiplier — 
inside — being horizontal AC a fundamental wave — a component - it is -- FA (1 0) -- a 
block - B - ' — and a block -- C — orthogonal transformation a multipUer - inside - 
being horizontal - AC - a fundamental wave -- a component -- it is -- FB -- ' (1 0) -- FC (1 0) 
" three - a sort a value size - relation -- evaluating . That is, the value of FA (1 0) 
judges whether it exists between the value of FB' (1 0), and the value of FC (1 0). If it exists, 
if it is no, it will move to S608 S609. 

[0075] Variable Vx is computed by the following formulas (10) S608. 
Vx=(FA-(l 0) FB' (1 0))/(FC-(l 0) FB' (1 0)) .... (10) 
Moreover, in S609, Variable Vx is set as Vx=0. 

[0076] the same S - 610 — **** — a block A orthogonal transformation a multiplier 

inside — a perpendicular direction AC — a fundamental wave -- a component -- it is 
FA (0 1) " a block — B — ' and -- a block — D — orthogonal transformation a multiplier 
inside -- being horizontal - AC a fundamental wave -- a component -- it is FB — ' (0 1) - 
FD (0 1) " three a sort size — relation - evaluating . That is, the value of FA (0 1) 
judges whether it exists between the value of FB* (0 1), and the value of FD (0 1). If it exists, 
if it is no, it will move to S611 S612. 

[0077] Variable Vy is computed by the following formulas (11) S611. 
Vy=(FA-(0 1) FB' (0 l))/(FD-(0 1) FB' (0 1)) .... (11) 
Moreover, in S612, Variable Vy is set up with Yy=0. 

[0078] S -- 613 " **** — a formula - (-- ten ~) — a formula - (-- 11 -) — having computed 
Vx " Vy " a radical - a block — A — from truly - having moved — ** judging having 
a block (block B" is called) -- a motion vector AB " as follows -- setting up -- ending . 
[0079] 

[Equation 8] 

[0080] That is, the motion vector from Block A to block B' is [0081]. 
[Equation 9] 

« 

[0082] Since it becomes, cxVx of a formula (12) and the term of dxVy serve as a vector 
component with resolving power higher than the distance between pixels. 
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[0083] As mentioned above, although the motion vector operation part 104 was described, 
as mentioned above, the motion vector operation part 104 of the gestalt of this operation 
has been two steps of processings. First, they are retrieval of the block considered to be 
spatial the nearest, next the guess of the minute amount of gaps from a block for which it 
asked. Although the guess which used two steps of orthogonal transformation multipUers 
is carried out with the gestalt of operation mentioned above, the Ist-step block retrieval 
may use the approach of guessing by the comparison of the pixel value within a block by 
simplification of processing, improvement in the speed, etc., without using an orthogonal 
transformation multiplier. 

[0084] Next, the data processing section 106 is explained using drawing 7 . In drawing 7 , 
the coordinate Management Department 701 manages whether the block of the frame H 
which is an object frame is equivalent to the location of the frame G throat which is a 
criteria frame according to the vector computed from the motion vector operation part 104. 
From this coordinate Management Department 701, the address whose performance index 
of a formula (9) mentioned above was min is outputted. 

[0085] The NxN blocking section 702 blocks the image of Frame H per NxN pixel. If the 
pixel value information on the block (an attention block is called) used inside [ motion 
vector operation part 104 ] the preceding paragraph is held, it is not necessary to perform 
this means in the data processing section 106 interior anew, 

[0086] Similarly, the NxN blocking section 703 performs blocking of the NxN pixel unit of 
Frame G based on the address received from the coordinate Management Department 701. 
If the pixel value information on the block (the error minimum block is called) with which 
the performance index became min among the blocks by which this means was also created 
and evaluated inside [ motion vector operation part 104 ] the preceding paragraph, and the 
block (a circumference block is called) located around the error minimum block is held, it is 
not necessary to carry out in the data processing section 106 interior anew, 
[0087] Now Block A and the error minimum block of as opposed to Block A for the attention 
block on Frame H on Frame G Block B*, Moreover, the inside of two sorts of blocks shifted 
and blocked for 1 pixel of every horizontal right and left on the basis of block B', The block 
by which it was estimated that a performance index was small is considered as Block C, 
and the block which was perpendicularly shifted and blocked for 1 pixel of every upper and 
lower sides on the basis of block B' and by which it was estimated that a performance index 
was small among two sorts of blocks is similarly considered as Block D. 
[0088] Moreover, the block which makes a zero the x-coordinate of the zero of Block C and 
the y-coordinate of the zero of Block D is considered as Block E. As for Block E, 1 pixel of 
level perpendiculars will have shifted at a time from block B'. 

[0089] The average-value calculation section 704 is a means to compute the average value 
of the pixel value within the block A which is an attention block. If the zero coordinate of 
Block A is set to (aO, bO), it is the average TA of Block A, It computes as follows. 
[0090] 

[Equation 10] 
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[0091] The average-value separation section 705 is the average value TA computed from 
each pixel within Block A. It is a means to dissociate, by subtracting. If the value after 
separation is set to gH (x y), it will be computed by the following formulas (15). 
gH =(x y) fH-(x y) TA .... (15) 

[0092] On the other hand, the average-value calculation section 706 computes block B' of 
Frame G, and the average value of each block of C, D, and E. If the zero coordinate of block 
B' is made into (a', b'), they are average TB* of each block, TC, TD, and TE. It is computed 
as follows, respectively. 
[0093] 

[Equation 11] 



[0094] As explanation of the flow chart of drawing 6 described, horizontally c and d 1 pixel 
of the whole right and left however, when it blocks by carrying out, [ ** ] When the block 
shifted rightward is estimated that the performance-index result which shows similarity 
with Block A is small and c= 1 and the block shifted leftward conversely are estimated that 
a performance-index result is small, to c=-l and this appearance In a down case, in a case 
d= 1 and above, it is d=-l in a vertical comparison. 

[0095] Moreover, block B' and since 4 blocks of Block C, Block D, and Block E have the 
large lap of a block, while it is 4 blocks, they may carry out average-value calculation only 
of one block, and may subtract, add and compute only the non-lap pixel of a block from the 
average value of the computed block about the remaining average value of 3 blocks. 
[0096] Next, the following operations are performed in the average permutation section 
707. 
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hH =(x y) gH+(x y) (1-Vx') -(1-Vy') -TB+ Vx'-(l-Vy') -TC+ (1-Vx') and Vy' - TD+Vx'-Vy' and 
TE.... (20) 

[0097] Here, Vx' and Vy' show the distance from the zero (a*, b') of block B' to a 
interpolation point. That is, it is very few when the coordinate of Vx and Vy which were 
computed by the formula (10) mentioned above and the formula (11) agrees completely on a 
desired interpolation point, the value of Vx and Vy which were computed in fact - being 
based it - interpolation point Vx* with the nearest distance, and Vy* it will interpolate 
upwards, 

[0098] The example of the physical relationship of Vx, Vy, Vx', and Vy' is illustrated to 
drawing 8 . - The mark is the interpolation point which should be carried out interpolation 
truly, in order for the sample point of Frame G, Vx which x mark computed by the formula 
(10) and the formula (11) from the zero coordinate (a*, b'), the point which left only Vy, and 
O mark to make resolution increase. Now, in the case of c= 1 and d= 1, the coordinate of 
this interpolation point becomes (a'+Vx*, b'+Vy'). This interpolation point turns into a zero 
of Block A, and is an arranging point. 

[0099] The formula (20) means block B' and having permuted by the average value of Block 
C, Block D, and Block E for the average value of Block A. And the average value to 
permute serves as a linearity operation with an average value of 4 blocks depending on the 
interpolation location of Block A. In other words, they are block B' on a criteria frame, and 
the thing which is going to change so that Block C, Block D, and Block E may be suited, 
and is going to use only AC component of Block A about DC component of Block A. 
[0100] As mentioned above, although the data processing section 106 was described, in the 
gestalt of this operation, it does not restrict to the example mentioned above. Block B' and 
since the lap of Block C, Block D, and Block E is large, each average to compute is 
considered also when there is no great difference. In that case, the simple method of adding 
only average TB [ of block B' ]' to gH (x y) is also possible. 

[0101] Drawing 9 is a flow chart which has been described above, which includes 
processing to motion vector calculation and arrangement focusing on the frame control 
section 106 and which shows the operations sequence of the repeat processing when using 
especially three or more multiple frames. S901 estimates edge reinforcement for every 
frame first by stored ** (n+1) firom the m-th fi-ame (m+n) to a frame eye. And the mutual 
comparison of them is carried out by S902. 

[0102] Then, the frame eye which has the maximum edge reinforcement by S903 (m+p) is 
set up as a fi-ame G. This is a criteria frame. Next, in S904, Variable s and Variable q are 
initialized to 0. Next, by S905, it judges whether Variable s is equal to p. This will judge no 
for whether the frame which is going to carry out current processing is a criteria frame. 
[0103] If the frame s processed now is not a criteria frame, in S906, it will judge whether q 
is equal to 0. This judges whether it is that the count of a repeat which is carrying out 
current processing is the 1st time. If q is equal to 0, Frame G will be arranged by S907 and 
Variable q will be counted up by S908. When judged with no by S906, processing is judged 
to be 2nd henceforth, and since the frame G which is already a criteria fi-ame is arranged. 
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S907 and S908 jump it. 

[0104] Then, in S909, a motion vector is computed between Frame G and a £rame (m+s) eye 
(it considers as Frame H). Next, after carrying out data processing of the frame H by S910, 
it arranges by S911. By S912, after counting up Variable s, it judges whether the count of a 
repeat is n times S913. In the case of no, it is judged that the frame which has not been 
processed yet is stored, and it repeats the same down stream processing also to return and 
other frames to S905. After arrangement is completed to all the stored frames, it is 
compounded by the image information of one sheet and processing is completed. 
[0105] As mentioned above, although a series of processings of the gestalt of this operation 
have been explained, the description of this invention is in the selection section 102. 
Therefore, the contents of the motion vector operation part 104, the data processing section 
106, and the arrangement processing section 107 grade are not Umited. The configuration 
which arranges the pixel value of each set elephant frame is also considered without the 
motion vector operation is naturally possible and also processing the data of an object 
frame also by the approach of not using orthogonal transformation. 

[0106] Moreover, the performance index of the edge reinforcement of a formula (7) is not 

* 

restricted to this. The following formulas (21) are considered as a modification of a formula 

(7). 

[0107] 

[Equation 12] 



[0108] In this case, the value behind an edge extract filter means counting the number of 
pixels which became beyond a certain threshold. The reinforcement of the edge of the 
whole image can fully be grasped also by the formula (21). Moreover, an edge extract filter 
may not be restricted to drawing 3 , either, and a filter with more strong noise resistance 
may be used. 

[0109] Moreover, the method with which evaluation of edge reinforcement does not use an 
edge extract filter, for example, the method judged based on the transform coefficient of the 
high frequency component of orthogonal transformation, is considered. In that case, it 
evaluates whether which frame of the power of a RF region is large, and the frame judged 
that power is the largest is set up as a criteria frame. 

[0110] Moreover, it may not restrict to this and the gestalt of this operation may estimate 
using the characteristic quantity of other images, although edge information was used for 
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* • 

the characteristic quantity of an image. 

[0111] Drawing 10 is a flow chart which shows the operations sequence by the gestalt of 
operation of the 2nd of this invention. It is only that the selection approaches by the 
selection section 102 of drawing 1 differ, and the gestalt of this operation is the same about 
other each part. Moreover, the flow chart of drawing 10 shows the example which creates 
the static image of high resolving of one sheet based on the image information of ** (n+1) 
from the m-th frame (m+n) to a frame eye. 

[0112] SlOOl shows a division process and substitutes the integral part at the time of doing 
the division of the value of n by 2 as p. On actual processing, it is realizable by the bit shift. 
Then, a frame (m+p) eye is set up by S1002 as a &ame G which is a criteria frame. Next, in 
S1003, Variable s and Variable q are initialized to 0. Next, by S1004, it judges whether 
Variable s is equal to p. This will judge no for whether the frame which is going to carry out 
current processing is a criteria frame. 

[0113] If the frame S processed now is not a criteria frame, in S1005, q will judge whether 
it is 0. The count which processes this now judges whether it is the 1st time. If q is 0, 
Frame G will be arranged by S1006 and Variable q will be counted up by S1007. When 
judged with no by S1005, processing is judged to be 2nd henceforth, and since the frame G 
which is already a criteria frame is arranged, S1006 and S1007 are jumped. 
[0114] Then, after computing a motion vector between Frame G and a frame (m+s) eye (it 
considers as Frame H) and carrying out data processing of the frame H by S1009, it 
arranges by SlOlO S1008. By SlOll, after counting up Variable s, it judges whether the 
count of a repeat is n times. When the frame which is not processed is still stored, the same 
down stream processing is repeated also to return and other frames to S1004. 
[0115] After arrangement is completed to all the stored frames, it is compounded as a 
single image and processing is completed. As mentioned above, it is the description that 
the gestalt of this operation opts for selection of a criteria frame by the order of a frame 
into which it was inputted. 

[0116] Drawing 11 is drawing having shown the decision of the criteria frame at the time of 
storing by five frames. The frame shown with the slash is a criteria frame. Since it is set to 
n= 4 when a storing image is five frames, by doing a division by 2, it is set to p= 2 and the 
intermediate frame of a frame (m+2) eye is set up as a criteria frame. This criteria frame 
will be processed [ other four frames / respectively ]. 

[0117] Although it cannot become in the middle correctly since the result of having done 
the division of the n by 2 becomes nonintegral when a storing frame number is even, the 
frame before and behind middle may be set as a criteria frame (however, in the flow chart 
of drawing 10 , it comes before middle). That is, with the gestalt of operation of the flow 
chart of drawing 9 mentioned above, it was the approach of setting up selection of a criteria 
frame based on "the description of an image." The characteristic quantity of an image 
evaluated the reinforcement of an edge as a performance index which can be expressed 
most notably. If surely it chooses in the description of an image, the image optimal in 
image quality may be able to be set up as a criteria frame. 
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[0118] However, since a sequential image is treated, it cannot time necessarily be said that 
it is the optimal. So, "the functionality of a time image" is thought as important and chosen 
with the gestalt of operation of the flow chart of drawing 10 . It can be assumed to be a 
central image with the highest functionality of an image to use a middle image in a 
time-axis if the continuity of an image is considered in comparison with each frame in a 
storing image. That is, since a time lag is min, there are few differences between each 
frame and a criteria image, and they end. 

[0119] As mentioned above, although the gestalt of operation of this invention was 
explained, the compromise of each flow chart of drawing 9 and drawing 10 is also 
considered. That is, it is also possible to create a new performance index in consideration of 
the characteristic quantity of an image and the location on a time-axis, and to determine a 
criteria frame. In that case, even when the optimal &ame is indistinct in image quality on 
a metaphor time-axis, the synthetically optimal image can be chosen. 

[0120] Next, the storage as a gestalt of other operations of this invention is explained. The 
purpose of this invention is possible also for attaining by the hardware configuration, and 
can also be attained by the computer system which consists of a CPU and memory. When it 
constitutes from a computer system, the above-mentioned memory constitutes the storage 
by this invention. That is, the purpose of this invention can be attained by using the 
storage which memorized the program code of software for performing actuation explained 
in the gestalt of each operation mentioned above with a system or equipment, and reading 
and performing the program code with which CPU of the system and equipment was stored 
in the above-mentioned storage. 

[0121] Moreover, as this storage, semiconductor memory, such as ROM and RAM, an 
optical disk, a magneto-optic disk, a magnetic medium, etc. may be used, and these may be 
constituted and used for CD-ROM, a floppy disk, a magnetic medium, a magnetic card, a 
non-volatile memory card, etc. 

[0122] Therefore, while a function equivalent to the gestalt of each above-mentioned 
implementation is realizable also by using with the alien systems and equipment other 
than the system which showed this storage to drawing 1 etc., or equipment, and reading 
and performing the program code with which that system or computer was stored in this 
storage, equivalent effectiveness can be acquired and the purpose of this invention can be 
attained. 

[0123] Moreover, when OS which is working on a computer performs a part or all of 
processing. Or after the program code read from the storage was written in the memory 
with which the extension unit coimected to the extension board inserted in the computer or 
the computer is equipped. Also when CPU with which the above-mentioned extension 
board and an extension unit are equipped performs a part or all of processing based on 
directions of the program code, while being able to reaUze a function equivalent to the 
gestalt of each above-mentioned implementation, equivalent effectiveness can be acquired 
and the purpose of this invention can be attained. 
[0124] 
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[Effect of the Invention] As explained above, even when are recording of the error at the 
time of vector calculation is not produced but the indistinct frame exists, according to this 
invention, satisfactory good composition is enabled by setting up the single criteria frame 
which serves as criteria of the comparison with each frame out of the stored multiple frame 
based on the characteristic quantity and the time functionality of an image. 
[0125] moreover, according to this invention, the image information which boiled markedly 
and was high-definition-ized compared with the interpolation of the high resolving still 
picture creation from the low resolving still picture of one sheet by which the conventional 
proposal was made, and a interpolation technique can be created. 

[0126] Further Since the high resolving still picture information on one sheet can be easily 
created from the low resolving still picture information photoed with the video camera 
according to this invention, the communication link between models &om which the 
resolution of I/O differs, a video camera, a printer which carry out expansion variable 
power and output a high definition image, etc. can be offered. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 1] It is the block diagram showing the image processing system by the gestalt 
of operation of this invention. 

[Drawing 2] It is the block diagram showing the selection section of drawing 1 . 
[Drawing 3] It is the block diagram showing the example of an edge extract filter. 
[Drawing 4] It is the block diagram showing the motion vector operation part of drawing 
1. 

[Drawing 5] It is a block diagram explaining a motion vector. 

[Drawing 6] It is the flow chart which shows the operations sequence of the transform 
coefficient evaluation section of drawing 4 . 

[Drawing 7] It is the block diagram showing the data processing section of drawing 1 . 
[Drawing 8] It is a block diagram explaining the arrangement location within a block. 
[Drawing 9] It is the flow chart which shows a series of processings including the 
selection section by the gestalt of operation of the 1 st of this invention. 
[Drawing 10] It is the flow chart which shows a series of processings including the 
selection section by the gestalt of operation of the 2nd of this invention. 
[Drawing 11] It is a block diagram explaining a criteria frame. 

[Drawing 12] It is a block diagram explaining the conventional maximum contiguity 
interpolation method. 

[Drawing 13] It is a block diagram explaining the conventional primary ** interpolation 
method. 

[Drawing 14] It is a block diagram explaining the conventional 3rd convolution 
interpolation method. 
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[Drawing 15] It is a block diagram explaining the comparison frame of the conventional 
motion vector calculation. 
[Description of Notations] 

102 Selection Section 

103 Frame Control Section 

104 Motion Vector Operation Part 

1 05 1 07 Arrangement processing section 

1 06 Data Processing Section 

108 Synthetic Section 

109 Interpolation Section 
201-204 Edge extract section 

205-208 Edge on-the-strength evaluation section 

209 The Maximum Edge on-the-Strength Decision Section 



[Translation done.] 
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